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Abstract. The results of Strassen [Str73| and Raz |RazlO| show that good enough tensor rank 
lower bounds have implications for algebraic circuit/formula lower bounds. 

We explore tensor rank lower and upper bounds, focusing on explicit tensors. For odd d, we 
construct field-independent explicit 0/1 tensors T : [n\'^ — >■ F with rank at least 2n^'^^'^^ + n — 
O(dlgn). This matches (over F2) or improves (all other fields) known lower bounds for d = 3 and 
improves (over any field) for odd d > 3. 

We also explore a generalization of permutation matrices, which we denote permutation tensors. 
We show, by counting, that there exists an order-3 permutation tensor with super-linear rank. We 
also explore a natural class of permutation tensors, which we call group tensors. For any group 
G, we define the group tensor Tq : G'' ^ F, by 7^(31, ... ,3^) = 1 iS gi ■ ■ ■ Qd = 1g- We give 
two upper bounds for the rank of these tensors. The first uses representation theory and works 
over large fields F, showing (among other things) that rankF(TQ) < |Gj'''''^. We also show that if 
this upper bound is tight, then super-linear tensor rank lower bounds would follow. The second 
upper bound uses interpolation and only works for abelian G, showing that over any field F that 
rankF(T'G) ^ 0(|G|^''''^ Ig''"^ |G[). In either case, this shows that many permutation tensors have 
far from maximal rank, which is very difi'erent from the matrix case and thus eliminates many 
natural candidates for high tensor rank. 

We also explore monotone tensor rank. We give explicit 0/1 tensors T : [n]'' F that have 
tensor rank at most dn but have monotone tensor rank exactly n"^"^. This is a nearly optimal 
separation. 
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1. Introduction 



Most real-world computing treats data as boolean, and thus made of bits. However, for some 
computational problems this viewpoint does not align with algorithm design. For example, the 
determinant is a polynomial, and computing it typically does not require knowledge of the under- 
lying bit representation and rather treats the inputs as numbers in some field. In such settings, 
it is natural to consider the computation of the determinant as computing a polynomial over the 
underlying field, as opposed to computing a boolean function. 

When computing polynomials, just as when computing boolean functions, there are many differ- 
ent models of computation to choose. The most general is the algebraic circuit model. Specifically, 
to compute a polynomial / over a field F in variables xi, . . . ,Xn, one defines a directed acyclic 
graph, with exactly n source nodes (each labeled with a distinct variable), a single sink node 
(which is thought of as the output), and internal nodes labeled with either +, meaning addition, or 
X , meaning multiplication. Further, each non-leaf is restricted to have at most two children nodes. 
Computation is defined in the natural way: each non-source node computes the (polynomial) func- 
tion of its children according to its label, and source nodes compute the variable they are labeled 
with. One can also consider the algebraic formula model, which requires the underlying graph to 
be a tree. In both of these models, we define the size of the circuit /formula to be the total number 
of nodes in the graph. 

Neither the algebraic circuit nor the formula model are well understood, in the sense that while it 
can be shown that there exist polynomials which require large circuits for their computation, no ex- 
phci£l examples of such polynomials are known. Indeed, finding such lower bounds for explicit func- 
tions is considered one of the most difficult problems in computational complexity theory. Several 
lower bounds are known, such as Strassen's |Str75j result (using the result of Baur-Strassen |BS83j ) 
that the degree n polynomial Y17=i -^F requires O(nlgn) size circuits. However, no super-linear size 
lower bounds are known for constant-degree polynomials. In the case of formulas, Kalorkoti |Kal85j 
proved a quadratic-size lower bound for an explicit function. 

One avenue for approaching improvements for both of these models is by proving lower bounds 
for tensor rank. A tensor is a generalization of a matrix, and an order-d tensor is defined as a 
function T : [n]'^ — )• F, where [n] denotes the set {1, . . . ,n}. A tensor is rank one if it can be 
factorized as T{ii, . . . = 11^=1 "^jih) "^j ^ T^^^ rank of a tensor is the minimum r such 
that T = Y2k=i '^k for rank one tensors Sk- It can be seen that an order-2 tensor is a matrix, and 
the notions of rank coincide. It can also be observed that the rank of an [n]*^ tensor is always at 
most n'^~^, and a counting-type argument shows that over any field there exist tensors of rank at 
least n'^~^/d. A tensor is called explicit if T(ii, . . . can be computed by algebraic circuits of size 
at most polynomial in poly((ilgn), that is, at most polynomial in the size of the input (ii, . . . ,irf). 
All explicit tensors in this paper will also be uniformly explicit. 

Interest in tensors arise from their natural correspondence with certain polynomials. Consider 
the sets of variables {Xij}i(z!^n],j&[d]- Given a tensor T : [n]*^ — >• F, one can define the polynomial 



This connection was used in the following two results. First, Strassen |Str73| showed that 



Theorem (Strassen |Str73j . see also [vzG88j ). For a tensor T : [n]^ — )• F, the circuit size complexity 
of fx is ri(rank(T)). 
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Thus, any super-linear lower-bound for order-3 tensor rank gives a super-linear lower bounds for 
general arithmetic circuits, even for the constant degree polynomials. More recently, Raz [RazlOj 
proved 

Theorem (Raz [RazlOj For a family of tensors Tn : [n]'^(") F with rank(r„) > n(i-°(i))'*(") 
and uj{l) < d(n) < O (log n/ log log n), the formula- size complexity of fT„ is super-polynomial. 

Thus, while Strassen's result cannot be used to prove super-quadratic circuit-size lower bounds 
(because of the upper bounds on order-3 tensor rank), Raz's result shows that tensor rank could be 
used to prove very strong lower-bounds. These results motivate a study of tensor rank as a model 
of computation in of itself. 



2. Prior Work 

Strassen's connection between order-3 tensor rank and circuit complexity further established a 
close connection between tensor rank to what is known as bilinear complexity. As several important 
problems, such as matrix multiplication and polynomial multiplication, are bilinear, one can study 
their bilinear complexity, and thus their order-3 tensor rank. We interpret various prior results 
in the language of tensor rank. For the matrix multiplication (which corresponds to a tensor 
of size [n^] x [n^] x [n^]), Shpilka |Shp03 showed that the tensor rank is at least 3n^ — o(n^) 
over F2, and Blaser [Bla99j earlier showed that over any field the tensor rank is at least 2.5n^ — 
0(n). For polynomial multiplication (which corresponds to a tensor of size [2n — 1] x [n] x [n]), 
Kaminski |Kam05j showed that the tensor rank over ¥g is known to be (3 -|- l/@{q^))n — o(n) and 
earlier work by Brown and Dobkin [BD80j showed that over F2 the tensor rank is at least 3.52n. 
Lower bounds for these problems seem difficult, in part because strong upper bounds exist for both 
matrix multiplication and polynomial multiplication. 

This work attempts to prove tensor rank lower bounds for any explicit function, not just problems 
of prior interest such as matrix or polynomial multiplication. Previous work in this realm include 
that of Ja'Ja' |Ja'79j (see their Theorem 3.6), who used the Kronecker theory of pencils to show 
tensor rank lower bounds of 1.5n for [n] x [n] x [2] tensors, for large fields. The work was later 
expanded by Sumi, Miyazaki, and Sakata [SMS09 ] to smaller fields. However, in these works the 
rank is shown to be at most 1.5n, so seemingly cannot be pushed further. 

It is also worth noting that Hastad proved [Has89l IHas90j that determining if the tensor rank 
of T : [n]^ — )• F is at most r is NP-hard, for F finite or the rationals (the problem is also known to 
be within NP for finite F, but not known for the rationals). Implicit in his work is a tensor rank 
lower bound (for explicit order-3 tensors) of 4n/3. To the best of our knowledge, the hardness of 
approximating tensor rank is an open question. Part of its difficultly is that any gap-preserving 
reduction from NP to tensor rank would automatically yield lower bounds for explicit tensors. 

It is also a folklore result (eg. see Raz [RazlOj ) that one can reshape, or embed, a n'-'^/^-l x n'-'^/^J 
size matrix into a order-d tensor, thus achieving a nL'^/^J rank lower bound for [n]^ size tensors. 



3. Our Results 

Our work has several components, each studying different aspects of the tensor rank problem. 
We first give two new methods in proving tensor rank lower bounds. In Section [5l we detail the first 
construction, which proves the best knowrQ tensor rank lower bound for a tensor of size [n] x [n] x [n] 
(over any field). In particular, using a generalization of Gaussian elimination we prove 



When comparing this result to those listed in the prior work, it is helpful to note the diflterences in size of the 
tensors, such as comparing [n^]^ (for matrix multiplication) to our [n]'^ ■ Thus, over F2, we essentially match Shpilka's 
3n^ — ci(n^) matrix multiplication result up to low-order terms. 
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Theorem (Corollary 15. 7p . Let¥ be an arbitrary field. There are explicit {0, l}-tensors Tn '■ [nf' — )• 
F such that rank(r„) = 3n — 0(lgn). 

However, our analysis of this construction is exact so no further improvements can be made. In 
Appendix ini we give a different order-3 tensor construction with a 3n — 0(lgn) rank lower bound 
over F2 that has no matching upper bound, and leave as a open question what is the correct rank. 
In Appendix[El we show how to extend the order-3 tensor rank lower bounds to yield a lower bound 
(for odd d) of 2nl-'^/'^-l + n — G(cilgn) for the tensor rank of an explicit 0/1 size [nY tensor, which 
is improves by a factor of 2 on the folklore reshaping lower bound of nL'^/^J. 

In Section [6l we explore the tensor rank of permutation tensors. For matrices, permutation 
matrices are all full-rank and have a tight connection with the determinant. Consequently, it is 
natural to conjecture that a generalization of permutation matrices, which we call permutation 
tensors, have high rank. In particular, using a counting lower bound for Latin squares, we show 
that indeed there is a order-3 permutation tensor with super-linear tensor rank (over finite fields). 

A natural class of permutation tensors are those constructed from groups. That is, for a fi- 
nite group G we can define the group tensor Tq : — t- F, which is a 0/1 tensor defined by 
T^{gi, . . . ,ga) = 1 iS gi ■ ■ ■ = Iq- It seems natural to conjecture that these tensors might also 
have high-rank. However, using representation theory we can give a strong upper bound on the 
rank of any group tensor (over large fields such as C). To prove results over any field, we use 
interpolation methods and field-transfer results to bound the rank of any group tensor arising from 
an abelian group. In particular, we have the following theorem. 

Theorem (Theorem 16.51 and Corollary 16. lip . Let G be a finite group. For "large" fields ¥, 
rankF(rJ) < IGj'^/^^ Further, for anyfieldF, ifG is abelian thenrankwiT^) < 0{\G\^+^^'^lg'^'^ \G\). 

In each case, we show that group tensors have rank far from the maximal Qd{\G\'^~^), and thus 
are not good candidates for high tensor rank for large d (which are needed for Raz's application). 
We are unable to place non-trivial upper bounds on the rank of Tq for G non-abelian and small F, 
but it seems natural to conjecture that strong upper-bounds exist given the above results. While 
these results do not unconditionally imply any circuit lower bounds, they elucidate differences 
between tensor rank and matrix rank by proving that group tensors are not a viable candidate of 
high-rank tensors. However, conditioned on the upper bound given in Theorem 16.51 being tight, we 
are able to give super-linear tensor rank lower bounds for explicit order-3 tensors. 

Finally, in Section [7] we explore monotone tensor rank. Monotone computation exploits the 
idea that if a polynomial only uses positive coefficients (over an ordered field such as Q), then 
one might try to compute this polynomial only using positive field elements. Previous researchers 
have tried, in various models, to show that such restricted computation is much more inefficient 
than unrestricted computation. Indeed, for general algebraic circuits Valiant |Val80j has shown 
that allowing negative field elements allows for an exponential improvement in the efficiency of 
computing certain polynomials. We continue in this line of work. In particular, we can show the 
following nearly optimal separation. 

Theorem (Theorem 17. 3p . Let F be any ordered field. There is a explicit 0/1 tensor T : [nY — F 
such that rankjr(r) < dn, but the monotone rank of T is n'^^^. 

4. Definitions and Notation 

We first define tensors, and give some basic facts about them. Throughout this paper, [n] 
shall denote the set {1, . . . ,n}, [nj shall denote the set {0, . . . ,n — 1}, and Ign shall denote the 
logarithm of n base 2. Further, the notation fEj (the Iverson bracket) will often be used as an 
indicator variable for the event E, and can be distinguished from [nj by context. 
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Definition 4.1. A tensor over a field F is a function T : 11^=1 [%] ~^ ^- is said to have order 
d and size (ni, . . . , n^). If all of the Uj are equal to n, then T is said to have size n'^. T is said to 
belong to the tensor product space (8)^^^F"j . 

In later sections, the input space of a tensor will sometimes be a group or a set \n\ instead of 
the set [n]. Throughout this paper F shall denote a arbitrary field, the variable n (or (m, . . . ,nd)) 
shall be reserved for the tensor size, and d shall be reserved for the order. Fg will denote the field 
on q elements. We can now define the notion of rank for tensors. 

Definition 4.2. A tensor T : Y['j=i[^i] — F is simple if for j G [d] there are vectors vj S F"^ such 
that T = ®'j^iVj. That is, for all ij G [n^], T(ii,...,irf) = 0^=1 ^i(b) where Vj{ij) denotes the 
ij-ih. coordinate of Vj. 

Definition 4.3. The rank of a tensor T : Y['j=i[^j] ~^ defined as the minimum number of 

terms in a summation of simple tensors expressing T, that is, 

rankF(r) = min |r : T = ^ ^j=iVj,k, Vj,k G F"^ | 

Notice that by definition, a non-zero tensor is simple iff it is of rank one. 

The next definition shows how identically sized order-((i — 1) tensors can be combined into an 
order-d tensor. 

Definition 4.4. For Ti, . . . , r„^ G «)J^{F"^ define T = [Ti\ ■ ■ ■ \TnJ by the equation T{ii, id-i,id) = 
Ti^(ii, . . . The Ti. are said to be the layers of T (along the d-th axis). Layers along other 

axes are defined analogously. 

Conversely, given T G ^'j^iF"'^ , define the l-th layer of T (along the d-axis), sometimes 

denoted T/ G (8'^~JF"j , to be the tensor defined hy Ti{ii, . . . , id-i) = T{ii, . . . , id-i, I)- 



5. COMBINATORIALLY-DEFINED TENSORS 



In this section, we construct combinatorially-defined tensors and prove linear lower bounds for 
their rank. To do so, we use the follow fact about tensors, which is proved in Appendix [Bj For 
matrices, this can be seen as a statement about Gaussian elimination. 



Corollary (Iterative Layer Reduction, Corollarv IB.2p . For layers Si, ... , G F"i (X" • • • (8) F"'*-i 
with Si, ... , Sm linearly independent (as vectors in the space F"^"'"'*-! there exist constants Cij G 
¥ , i £ {1, . . . , m}, j G {m + 1, . . . , rid}, such that 



(5.1) rank([Si| 



\Sna])> rank 



Sn 



-1 + ^ Ci^m+lSi 



Ci 



5,: 



i=l 



+ m 



The idea of this section is to construct tensors such that we can apply Corollary IB. 21 as many 
times as possible. As mentioned in Remark IB. 71 for a [rif tensor, the lemma can be applied at most 
dn times, and thus the lower bounds can at best be dn. In general, the lemma may not be able to 
be applied this much because the elimination of layers zeroes out too much of the tensor. However, 
in this section we construct tensors (for d = 3) such that we can almost apply the lemma dn times. 
The result is that we give explicit (order 3) 0/1-tensors with tensor rank exactly 3n — 0(lgn) 
over any field. To begin, we apply the above corollary twice, along two different axes, to get the 
following lemma. A full proof of this lemma, along with other claims in this section, can be found 
in Appendix O 
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Lemma 5.2. Let Ai, . . . , be n x n sized ¥ -matrices. Let 1^ denote the m x m identity matrix, 
and Om denote the m x m zero matrix. Then, 



(5.3) 

and 

(5.4) 
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> rank([Ai| • • • 1^^]) + 2n 



> rank([Ai| • • • 1^^]) + 2n 



where the left-hand side of Equation \5.4\ expresses the tensor rank of a [2n + 1] x [2n + 1] x [k]-sized 
tensor. 

Applying this lemma recursively yields the following construction. 

Definition 5.5. Let : N — )■ N denote the Hamming weight function. That is, H{n) is the number 
of I's in the binary expansion of n. 

Theorem 5.6. For i G {0, . . . , [IgnJ}, let Sn,i be an nxn matrix defined in the following recursive 
manner. 

• 5i,o = [1] 

• For 2n > 1, 



'2n,i 



On On 
Sn,i On 
In On 
On In 



if i < [Ig nj 
if i = [Ig nJ 



For 2n + 1 > 1, 



S' 



2n+l,i 





On On ifi < [lgn\ 

_Sn,i On 

"o o] 

In On ifi= [Ig '^J 

_0n In 0_ 

Then, denoting r„ = [5n,o| • • • |5'„,Lignj]; 

(1) Tn has size [n] x [n] x [[IgnJ + 1]. 

(2) rank(Tn) = 2n - 2H{n) + 1 . 

(3) On inputs n and {i,j,k) £ [n] x [n] x [[IgnJ + 1], Tn{i,j,k) can be computed in polynomial 
time. That is, in time 0(polylog(n)). 

Another application of Corollary IB. 21 (along the one axis it has not yet been applied) yields the 
following claim. 

Corollary 5.7. Define Sn,i as in Theorem \5.6l Let n € N and restrict to n >2. Then, for i G [n] 
define nxn matrices S'^ ^ by 



Sn-l,i-l 



On-1 ej_Qig(„_i)j_|_i) 
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z/zG [Llg(n-l)J +1] 
else 



where ej E F" ^ is the indicator column vector where ej{k) = [j = kj. (Notice that [lg(n — 1)J +1 < 
n — 1 for all n > 2.) Then, denoting T'^ = [S'^ il " " " {S'n n]: 

(1) has size [n]^. 

(2) rank(r;) = 3n - 2H{n - 1) - Llg("- - 1)J - 2 > 3n - e(lgn). 

(3) On inputs n and {i,j,k) G [n] x [n] x [n], T!^{i,j,k) can he computed in polynomial time, 
that is, 0(polylog(n)). 

Note that this analysis is exact. Appendix|D] contains similar lower bounds over F2 using different 
methods, where no non-trivial upper bound is known. 

6. Permutation Tensors 

One of the most natural families of full-rank matrices are permutation matrices. This section 
examines a natural generalization of permutation matrices to tensors, which we call permutation 
tensors. A counting argument (Proposition 16. 2|) shows that there exists order-3 permutation ten- 
sors of super-linear rank (over any fixed finite field), so it is natural to conjecture that permutation 
tensors may all have near-maximal rank, just as in the matrix setting. However, we show (Sub- 
section 16. 2p that this is false: we give tensor rank upper bounds proving that permutation tensors 
constructed from groups have rank much less than maximal. 

We begin with the formal definition of permutation tensors. 

Definition 6.1. Let F be a field, and T be a tensor T : [nf T is a permutation tensor if 

T assumes only 0/1 values, and T has exactly one 1 in each generalized row. (A generalized row, 
sometimes just "row" , is the set of n inputs to T resulting from fixing d — 1 of the coordinates, and 
varying the remaining coordinates). 

It is not hard to see that order-2 permutation tensors and permutation matrices; as permutation 
matrices are those 0/1-matrices such that each row and column have exactly one 1. 

6.1. Permutation Tensors: Rank Lower Bounds. We now show that there exist permutation 
tensors of super-linear rank (over finite fields). 

Proposition 6.2. Let ¥ be a finite field. Then there exists a permutation tensor T : [n]^ — t- F 0/ 
rank at least il(nlog|ip| n). 

Proof. A Latin square is an n x n matrix, with each entry labeled with a symbol from [n], such that 
no symbol is duplicated in any row or column. Observe that order-3 permutation tensors exactly 
correspond to Latin squares. We now use the following fact about Latin squares, whose proof uses 
lower bounds for the permanent of doubly-stochastic matrices. 

Theorem ( [vLWOl] ). The number of n x n Latin squares is at least (n!)^"/n'^^. 

A standard counting argument completes the claim. □ 

It remains unclear if this result generalizes to higher orders. That is, can one show that for any 
k > 3 there exist permutation tensors of rank at least a;(nL'^/^J )? 

6.2. Permutation Tensors: Rank Upper Bounds. In this section we define a class of per- 
mutation tensors constructed from finite groups, and show that these tensors have rank far from 
maximal. We will give two rank upper-bound methods. The first method uses representation the- 
ory and accordingly only works where the group has a complete set of irreducible representations 
(which usually means "large" fields). The second method is based on polynomial interpolation, 
and while it gives worse upper bounds and only works for finite abelian groups, it gives results over 
any field. Neither of these methods applies to all finite non-abelian groups over small fields, and 
the rank of the corresponding tensors is unclear. 
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Definition 6.3. Let G be a finite group (written multiplicatively, with identity Iq) , and F a field. 
Define the order-A; group tensor Tq : — ?■ F by 

Tcigi, ■■■,gd) = l9i---gd = icl 

We first explore the representation-theory based upper bound. To do so, we first cite relevant 
facts from representation theory. 

Theorem 6.4 ( |Ser77j ). Let G be a finite group and F a field. A representation of G is a 

homomorphism p : G ^ prfxd^ where d is the dimension of the representation and is denoted 
dim p. The character of a representation p is a map Xp ■ G ^ ¥ defined by tr op, that is, 
taking the trace of the resulting matrix of the representation. 

//'char(F) is coprime to \G\, and¥ contains N-th roots of unity, for N equal to the least common 
multiple of all of the orders of elements of G, then there exists a complete set of irreducible 
representations. In particular, for c denoting the number of conjugacy classes of G, there is a set 
of representations pi, . . . ,pc and associated characters such that (among other properties) we have 

(1) ^n=i(dimp.)-X.(5) = Iff = el 

(2) ELi(dimp.)'=n 

(3) dim Pi divides \G\ 

In particular, for finite abelian groups, c = n and dim. pi = 1 for all i. 

Notice that property (1) in the above theorem is an instance of the column orthonormality 
relations of character tables, which follow from the more commonly mentioned row orthonormality 
relations. We now use these facts to derive upper bounds on the rank of Tq when the conditions 
to the above theorem hold. 

Theorem 6.5. Let G be a finite group, d >2 and¥ afield, such that char(F) is coprime to \G\, and 
¥ contains N-th roots of unity, for N equal to the least common multiple of the orders of elements 
of G. Then given the irreducible representations pi,. . . ,pc for G over ¥, the order-d group tensor 
has \G\ < rankF(r^) < X]-=i(dimpi)'^ < \G\'^/^ . 

In particular, for finite abelian groups, rank^(TQ) = |G|. 

Proof. rankF(T^) > |G|: This follows from observing that for fixed gs, ■ ■ ■ ,gd, Tq{-, -,5(3, . . . ,gd) is a 
permutation matrix, and thus its rank (of |G|) lower bounds the rank of Tq (over any field). This 
can also be seen by induction on Corollary lA. 91 

G abelian =^ rankir(T(^) < |G|: Theorem 16.41 further implies dimpj = 1 for all irreducible rep- 
resentations of finite abelian groups, which implies rankF(Tg) < |G| for abelian groups. 

X]i=i(dimpj)'^ < Theorem l6.4r 2) shows that X]i=i(di™/Oi)^ = ^- Thus the claim is equiv- 

alent to showing that for d £ Z,, d > 2 and G M>0) "^ni = n =^ ^f^^ — ^'^^^^ To show 
this, we first show that (n + m)"'/^ _^ Qd/2 = + m)°'/^ > n^^/^ _^ ^d/2 _ 

see, this, observe that 

assuming without loss of generality that n > m, we have that 



where we use that (|-^/2]) — — 2. Taking square roots yields (n + m)'^/^ + 0^/^ > n'^^'^ + m'^l'^ . 
Thus, given non-negative rij summing to n, one can iteratively zero out certain Ui while increasing 
the sum ^Y^r^J"^ ., until only n\ = n and thus Xl^-i^^^ — n-'^^'^- Thus, this is a bound on the initial 

r d/2 

sum of 2^ rij . 

rankF(T^) < X^i=i(dimpj)°': The result will follow by constructing, for each i, the order-d tensor 



Tp^gi, ■■■,9d)= Xi{gi 



■9d) 



in rank (dim/Oj)'^. Theorem I6.4f 1) shows that Tq = |^ ^^=x(dim/9j) • Tp_ and so distributing the 
dim/9j/|G| term inside the simple tensors yields the result (where we crucially use the restriction 
on the field characteristic). 

Thus, all that remains is to show that rankp (T^. ) < (dim/jj)'^. Using the group homomorphism 
properties of the representations and expanding the definition of the trace through the matrix 
multiplication we see 

dim pi dim pi 

Tp^{gi,...,gd)= ^ ••• ^ {pi{9i))k^M - ■ ■ iPii9d~i))ki_^,ki{pii9d))kiM 

ki=l ka=l 

and one can observe that for fixed ki,...,kd, the function {pi{gi))ki,k2 ■ ■ ■ {Pi{9d~i))ka-i,kaiPi{9d))kaM 
is a simple tensor so the above shows rankF(Tp.) < (dim pi)'^ as desired. □ 

The above result is possibly tight, motivating the question: is there a group G and irreducible 
representation p oi G such that rankF(T^) < (dimp)'^? As the above result is tight for abelian 
groups, any affirmative answer to the above question would involve a non-abelian G. 

Even supposing the above result was tight, one can ask what implications this gives for circuit 
lower bounds, especially because group tensors are explicit when the defining group operation is 
efficiently computable. However, applying tensor rank lower bounds to Raz's [RazlOj result requires 
order-d tensors of rank n^^~°^^^^'^, and Theorem 16.51 shows that no group tensor can achieve this 
rank over large fields. In particular, for the purposes of tensor rank lower bounds, the lower bounds 
of Corollary IE . 21 are asymptotically (in d) as good as the rank achievable by any group tensor (over 
large fields). 

However, if tight. Theorem 16.51 would yield better lower bounds than Corollary IE. 21 for odd d. 
In particular, the symmetric group ©„, has a complete set of irreducible representations over the 
rationals [Ser77j . Thus, the tightness of Theorem 16.51 would imply a lower bound for rank(Q(Tg J, 
which is an explicit tensor. To understand this lower bound, the following fact is useful. 

Theorem 6.6 ( |VK85j ). The largest dimension of an irreducible representation of (5„ over Q is 
of size \/^/e®(v^) 

In particular, for d = 3, all of the above imply that rankQ(T|g ) > |0„|-'^'^/e , which is 

r2(n^'^^'^), for any e > 0. Then, applying Strassen's |Str73j result would yield 0(n^'^~'^) lower bounds 
for the (unrestricted) circuit size of explicit degree-3 polynomials (that have 0/1 coefficients). Such 
a conclusion would surpass the best known circuit size lower bound even for super-constant degree 
polynomials, which is Strassen's |Str75] r2(n log n) lower bound for degree n polynomials. Thus, 
tightness of Theorem 16.51 would have interesting consequences. 

Regardless of whether the result is tight. Theorem [62] only works over "large fields" in general. In 
particular, it does not (in general) give insight into the rank of group tensors over fixed finite fields, 
or even over the rationals. To take an example, the cyclic group Z„ requires n-th roots of unity for 
its irreducible representations. While Lemma 16.101 does show a relation between rankQ(r^^) and 
rankQjj.]/^^:"-!) (^z ) (where Q[x]/{x"^ — 1) is the field of rationals adjoined with a n-th primitive 
root of unity, so Theorem 16.51 applies) this relationship implies nothing beyond trivial rank upper 
bounds. Thus, to achieve rank upper bounds for group tensors over small fields we take a different 
approach, one using polynomial interpolation. Our result only applies to finite abelian groups, but 
is able to show that have "low" rank in this regime. 
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Proposition 6.7. Let ¥ be a field with at least d{n— 1) + 1 elements. Let T : [nj'^ — )• F 6e a tensor 
such that 

d(n-l) 

T{ii,. . . ,id) = ^ Cmlii+i2-\ \-id = mj 

m=0 

for constants Cm G F. Then, rank(T) < d{n — 1) + 1. 

Proof Sketch. We sketch the proof here, the full proof is in Appendix|Fl The proof follows the result 
of Ben-Or (as reported in Shpilka-Wigderson [SWOlj ) on computing the symmetric polynomials 
efficiently over large fields. To compute a desired polynomial /(x), one can introduce a new variable 
a and an auxiliary polynomial P{a, x) such that 

• P is efficiently computable, of degree at most d' in a 

• For some m, f{x) = Cam{P{a,x)). That is, / equals the coefficient of a™ in P. 

To compute / on input x, we can then evaluate P on (ai, x), . . . , {ad'+i,x) and then use interpo- 
lation to recover C^m = f{x). 

To apply this idea to tensors, we observe the coefficients (in the variable a) of the polynomial 

P{a, {Xf},,) := n (X« + aXf ) + a^X^ + • • • + a'-^X^ i) 

i=l 

exactly correspond to the type of tensors we are trying to produce. As P is degree at most d{n — 1) 
in a, and further P is a rank one tensor in disguise, interpolation completes the result. □ 

We now turn to using Proposition 16.71 to upper bound the rank group tensors formed from cyclic 
groups. 

Corollary 6.8. Let¥ be a field with at least (i(n — 1) + 1 elements. Then, rank(r^^) < d{n — l) + l. 

Using the Structure Theorem of Abelian Groups the following can now be shown (for proof see 
Appendix [F]) . 

Corollary 6.9. Let G be a finite abelian group, and ¥ be a field with at least \G\ elements. Then 
rankF(T^) < \G\^+^^'^. 

While all of the results based on Proposition 16.71 do not require the field to have large roots 
of unity, they still require the field to have large size. Thus, they seemingly do not answer the 
question of the rank of group tensors over small fields. However, as the next lemma shows (with 
proof in Appendix |F]), one can transfer results over large-sized fields to small-sized fields with a 
minor overhead. 

Lemma 6.10. Let K a field that extends ¥. Then for any tensor T : [n]*^ — )• F, rankF(T) < 
(dimp IK)'^^-'^ • rankK(T), where dimflK is the dimension o/K as an ¥-vector space. 

With this field-transfer result, we can now state rank upper bounds for group tensors (for finite 
abelian groups) for any field. 

Corollary 6.11. Let ¥ be any field, andG be a finite abelian group. Thenrankw{T^) < \G\'^+^^'^\lg\G\Y~'^ ■ 
In particular, if G is cyclic, i/ien rankF(T^) < (i|G| [Ig 

This last result shows that for any finite abelian group, any field and any large d, the rank of 
the corresponding group tensor is far from possible Q{n'^~^). These results do not settle the rank 
of group tensors for non-abelian groups over small fields, and leaves the open question whether the 
methods of Theorem 16.51 or Proposition 16.71 (or other methods) can resolve this case. 
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7. Monotone Tensor Rank 



We now explore a restricted notion of tensor rank, that of monotone tensor rank. In algebraic 
models of computation, monotone computation requires that the underlying field is ordered, which 
we now define. 

Definition 7.1. Let F be a field. F is ordered if there is a linear order < such that 

• For all x,y,z £¥, X < y =^ x + z < x + y. 

• For all x,y £¥ and z G F>o, x < y =^ xz < yz. 
where F>o = {x\x G F, x > 0}. 

Recall that every ordered field has characteristic zero, and thus is infinite. 

Over ordered fields, computation of polynomials that only use positive coefficients can be done 
using only positive field constants, but many works (such as [ValSOj ) have shown that the circuit 
model of computation, the restriction to positive field constants in computation leads drastically 
worse efficiency as compared to unrestricted computation. In this section, we show that in the tensor 
rank model of computation, monotone computation is also much less efficient then unrestricted 
computation. We first define the notion of monotone tensor rank. 

Definition 7.2. Let F be a ordered field. Consider a tensor T : HiLii"-?] ~^ ^>o- Define the 
monotone tensor rank of T, denoted m-rank(T), to be 

m-rank(T) = min |r : T = ^ v^^i (g) • • • (g) vi^d, vi^i G (F>o)"'' | 

We now show an essentially maximal separation between monotone tensor rank and unrestricted 
tensor rank, for the explicit group tensor T^^. 

Theorem 7.3. Let F be an ordered field. Consider the group tensor . Then 

(1) mnk^{Tl)<d{n-l) + l 

(2) m-rankF(r|^) = n'^''^ 

Proof Sketch, see AppendixlG\ for a full proof . The upper bounds follow from Corollary 16. 8| and 
from the trivial n'^^^ upper bound for tensor rank. 

The lower bounds follows from the observation that a non-negative simple tensor "covers" non- 
zero entries in T^^ . It is not hard to show that if a simple tensor covers at least two non-zero entries 
in T^^ then it places a positive weight on a zero-entry of T^^. As this cannot be canceled out in 
a monotone computation, each simple tensor must cover at most one non-zero entry. As there are 
n'^~^ such entries, the result follows. □ 
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Appendix A. Basic Facts about Tensors 



We now prove some relevant facts about tensors that are needed for the rest of the paper. 
Lemma A.l. A ^j^^F""^ -tensor is an Y['j=i^j dimensional ¥-vector space, with standard basis 
{<^j=ieij j}ije[nj] where {eijj}i.(z[n^] is the standard basis for W^K 

Proof. Recall that the tensor product space is the set of functions from 17^=1 [^j] ^• 

As a F- valued function space, it is thus an F- vector space. That it is 11^=1 ''^h dimensional follows 
from the fact that this is the cardinality of the domain. 

To see that the basis is as claimed, note that the function ^j^iCi-j is equal to the tensor 

T{i'i, . . . ,i'^) = Y['j=i¥''j — It is then not hard to see that these tensors are a basis for the tensor 
product space. □ 

Lemma A. 2 (Multilinearity of Tensor Product). Suppose j G [d], vj G F"^, and a,6 G F. In the 
tensor product space ^j^-^W"'^ , for any jo G [d] and w G F^^o the following identity holds: 

vi ■ ■ ■ {avjg + bw) ■ ■ ■ Vd = a{vi ■ ■ ■ (S> vjo ■ ■ ■ (E> Vd) + K^i ^ ■ ■ ■ (E> w (E> ■ ■ ■ (E> Vj) 
Proof. This follows directly from Definition 14.21 □ 
We now use these properties to establish a class of rank-preserving maps on tensors. 

Lemma A. 3. For j G [d], consider linear maps Aj : F"^ — t- F^j . 

(1) The Aj induce a function on simple tensors ^j^^Vd i— )• 0j^iAjVj which uniquely extends to 
a linear map on the tensor product spaces which is denoted (8)^=1^4^ : (8)^^-^F"j — )• (g)^^;^F"j . 

(2) // the Aj are invertible, then so is 0j^iAj and its inverse is given by (^'j=iAj^ . 

(3) For T : 11^=1 [^j]'' ~^ rank(T) > rank(((g)^^]^Aj)(T)) , with equality if the Aj are invert- 
ible. 

Proof. ([1]): By Lemma [A. li the tensor product space (E>j=x^"^ ^ basis consisting entire of simple 
tensors. Thus by standard linear algebra, the map (^j^iVj ^ (E>j=iAjVj on this basis extends 



uniquely to a linear map 0j=iAj on the entire tensor product space. 



It must also be shown that the map 0j^iAj induced from the basis elements is also compatible 
with the map 0j^iVj i— ?• ^j=iAjVj defined on the rest of the simple tensors. This fact follows from 
the linearity of the Aj and the multilinearity of the tensor product, Lemma lA. 21 That is, we first 
use that each Vj can be expressed in terms of the basis elements Vj = Ylij=i'^ij,j^ij,j then 
notice that by multilinearity of the tensor product we have 




We observe similarly that ®j^iVj = YL^l=i ' ' ' SijLi • • • Ci^^d{® j=i(iij,j)- As the unique linear 
map induced above defines ^'^j^-^Aj{®j^^Vj) as ^"1=1 ' ' ' Erj^=i • • • (8)j=i ^i(®j=ieij j), 
this shows that ®'j=iAjVj = (S)j=i ^^(^5^=1^^^), and so the two maps agree on the simple tensors. 
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It should also be noted that this argument is independent of the basis chosen, as long as the 
basis is chosen among the simple tensors. This fact follows from the fact that the induced map on 
the entire space agrees with the map only defined on the simple tensors. Thus, the map (2)j=i 
is well-defined. 

([2]): Denote the linear maps A := CSj^i^j, and A^^ := tE)'j^iAj^ . Part [T] of this lemma shows 

that the maps A and A~^ compose, in either order, to be the identity on the simple tensors. As 
there is a basis among the simple tensors, by Lemma lA.ll this means that A o A^^ and A^^ o A 
are both identity maps. Thus A~^ is indeed the inverse map of A. 

Consider a minimal simple tensor decomposition of T, so that T = X][=i '^j=i^i,'- 
part m of this lemma, we have a simple tensor decomposition {^j=iAj)T = Yl\=i ®j=i^j^j,i- This 
establishes the desired rank inequality. To establish equality when the Aj are invertible it is enough 
to run the inequality in the opposite direction using the linear map 'S!)'j=iAj^ and using part [2] of 
this lemma. □ 

We now use these rank-preserving maps to establish facts about tensors and their layers. 

Lemma A. 4. Consider T = CS)'j^iVj € ^'j=i where T is split into layers as T = [Ti| • • • |T„,^]. 
Then Ti = in ■ ■ ■ Vd-i) ■ Vd{l) G F"^ . 

Proof. Definition l4.4l and Definition 14. 2 1 show that T;(ii ) := T{ii^ . . . ^i^-i^l) = vi{ii) ■ ■ ■ Vd~i{id-i) 

Vd{l)- We can then note that this is exactly the function vi® ■ ■ ■ ® Vd-i, multiplied by the scalar 
Vdil), as desired. □ 



Lemma A. 5. Consider the operation of taking the l-th layer (along the d-th axis). This is a linear 
map Li : 0^1^ F"^ ^ (g)^t! F"^ • 



Proof. Given the tensor T(-, ...,•). Taking the l-ih. layer yields T;(-, ...,•) := T(-, Z). Thus, 
the statements T = S + R =^ Ti = Si + Ri, and c G ¥,T = cS =^ T/ = cSi hold because they 
are simply a restriction of the above identity. □ 

We can now prove the main lemma of this appendix, on how applying linear maps interacts with 
the layers of a tensor. 

Lemma A. 6. Consider T e '^'j^iF"'^ . Expand T into layers, so T = [Ti\--- |r„^]. 

Let {aij)ij £ ^'"^"d 5e a matrix. Define A : F"'* — )• F™ to be the linear map the matrix 
induces via the standard basis. Then, 



{I ■ ■ ■ (g) 1 A){T) 



rid 



Ln=i 



rid 

—1 



Proof. The proof is in two parts. The first part proves the claim for simple tensors, and the second 
part extends the claim, using the linearity shown in Lemma lA.51 to general case. 

We first prove the claim for simple tensors. Let T = ^j^^Vj be a simple tensor. Let {ei^d}i£[nd] 
be the standard basis for F"'' and {ei'^d}i'e[m] be the standard basis for F"*. Then by expanding 
out in terms of the basis elements and using multilinearity, we have 

T = vi (g) ■ ■ ■ (g Vd 

rid 

= E (g) • • • Vd-1 ® {vd{i)ei,d) 

i=l 
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Denote T' := {I i 



I®A){T). So then, 

(g) Vd~i ® A{vd{i)ei^d) 

(g) Vd-i ® {vd{i) ■ A{ei^d)) 

/ m 



ltd 

T' = Yvi® 

1=1 
"d 

= ^ i;i O 

i=l 

i=l 

nd m 

= ^ ^ Vd{i) ■ a-i'/L ■ {vi®---® Vd-1 eii,d) 

i=l i'=l 



Vd-1 



i'=i 



By Lemma lA.41 and Lemma lA.51 we have, 

nd "-'d 



and using that ej/^rf(Z) = Ji' = 

= "^Vdii) ■ ai.i ■ {vi 



i=l i'=l 



rid 



ai,iTi 



i=l 



which establishes the claim for simple tensors. 

Now let T G ®'j^]¥^^ be an arbitrary tensor. Consider a simple tensor expansion T = X]fc=i "^fc 
for S'fc = 0j^iVj^k- Denote Sk^i to be the l-th layer of Sk- So then as the are simple, we have 



that {I < 
then. 



/ (g) A){Sk) = [Yl'ii=i ai,iiSk,h [■■ \Ta.^=i am,i^Sk^i^ ] by the above analysis. So 



{I' 



I®A){T) = (/. 



k=l 



I®A)[Y,Sk 



\k=l 



nd 



.jl=l 



nd 



by linearity of taking layers, Lemma lA.51 we get 

r na 

Lfe=i n=i 

ii=l \k=l 
nd 



r na 



k=l im=l 
nd 



.ii = l 



nd 

E 



^=1 



which is the desired result. 



□ 
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We now apply this to get a symmetry lemma. 

Corollary A. 7. Consider T G ^'^^^^^ . Expand T into layers, so T = [Ti| • • • |T„^]. For any 
permutation a : [n^] — )• [n^], 

rank([T^(i)| • • • |r^(„^)]) = rank([ri| • • • |r„J) 

Proof. Let P be the linear transformation defined by the permutation that a induces on the basis 
vectors of F"''. Then P is invertible, and so by Lemma lA.3121 the induced transformation I ® ■ ■ ■ ® 
I ® P \s also invertible and so rank((/ ® ■ ■ ■ ® I ® P)(T)) = rank(r) by Lemma lA.3l2i Then, by 
LemmaEH we see that (/ • • • / P){T) = [Ti] • • • |T„J. □ 

We also need another symmetry lemma. 

Lemma A. 8. For T G (g)^^]^F"j and a permutation a : [d] ^ [d], define T' G (g)^^j^F"''^(^) hy 
T'{ii,. ..,id)= r(v-i(i), . . . , v-i(rf)). Then, rank(r) = rank(r'). 

Proof. We show rank(T) > rank(T'), and the equality follows by symmetry as a is invertible. 
Consider a simple tensor decomposition T = Ylk=i^'j=i^j,k- It is then easy to see that T' = 
Y7k=i'^''^(T(j),k by considering the equation pointwise: T'{ii, . . . ,i(i) = T(zo— . . . , i^— = 

Z^fc=i nj'=i = Z]fc=i nj=i ^(T(j),fc(*i)- The conclusion then follows by considering a 

minimal rank expansion. □ 

Finally, we need a corollary about how dropping layers from a tensor affects rank. 
Corollary A. 9. For layers Si,... , 5„^, S' G <S)'jZi we have that 

rank([S'i| • • • jS'nJ) < rank([5i| • • • \SnJS']) 
with equality if S' is the zero layer. 

Proof. (<): The projection map P induces the map (/ (g) • • • (g) / (g) P) which takes [jSiI • • • |5n^|5"] 
to [^il • • • \Sn^] by Lemma lA.51 and so Lemma lA.3131 implies that the rank has not increased. 

(>): So now assume S' is the zero layer. Then again we apply Lemma's lA.5l and Lemma fA. 3131 but 
now extend the natural inclusion map l 

■ ]pna _^ fna+i ^ linear map (/(g - • - (g/fgit) on the tensors 
which takes [^il • • • {Sn^] to [5*11 •• • jSn^lO], again showing that the rank has not increased. □ 

Appendix B. Layer Reduction 

This section details a generalization of row-reduction, which we call layer-reduction. We show 
that layer-reduction can alter a tensor in such a way to provably reduce its rank. By showing this 
process can be repeated many times, a rank lower bound can be established. 

The following lemma is the main technical part of this section. Hastad implicitly usecfl a version 
of this lemma in his proof that tensor rank is NP-Complete |Has89l IHasQOj However, Hastad's 
usage requires that S-j^^ is a rank-one tensor. This special case does not seem to directly imply our 
lemma, which was independently proven. While the special case is sufficient to lower-bound the 
combinatorially-constructed tensors of Section [5l the full lemma is needed to lower-bound the rank 
of the algebraically-constructed tensors of Section iDl 

Lemma B.l (Layer Reduction). For layers Si, ... , G (E)j=i ^"'^ ""^^^^ non-zero, there exist 
constants ci, . . . , c^^-i G F such that 

rank([S'i| • • • \SnJ) > rank([S'i + ciSnJ ■ ■ ■ \Sna-i + Cna-iSnJ) + 1 

■^Hastad's usage, and proof, is reflected by Lemmas 2, 3 and 4 (and the following discussion) of the conference 
version [Has89|. The journal version [Has90| ascribes the origin of these lemmas to Lemma 2 in the work of Hopcroft 
and Kerr |HK71| 
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Proof. Denote T := [S*!! . . . 15'^^]. The proof is in two steps. The first step defines a hnear trans- 
formation A on F"'* such that the hnear transformation I ■ ■ ■ ^ I A is a higher-dimensional 
analogue of a row-reduction step in Gaussian elimination. That is, for T' the image of T, it is 
seen that T' = [Si + ciSnJ ■ ■ ■ \Sna-i + '^rid-i'^nJSnJ by Lemma \AM The q are chosen in such 
a way so that T' has a minimal simple tensor expansion where some simple tensor R is non-zero 
only on the 5„^-layer. In the second step, the 5n^-layer is dropped and the remaining tensor 
T" = [Si + ciSnJ ■ ■ ■ 1 5*11^-1 + Cn^-i'S'nd] no longer requires R in its simple tensor expansion and so 
rank(r) > rank(r") + 1. 

Consider a minimal simple tensor expansion T = X]fc=i ®j=i'^j,k- Expanding the Vd^k in terms 
of basis vectors yields 



r 

T = ^{vi^k ® • • • ® Vd-i,k) ® (^V,fc(l) • ei,d H h Vd,k{nd) ■ 4^,^)) 

k=l 



and in particular Lemma IA.4I shows that Sn^ = X]fc=i(^i,fc ■ ■ ■ ® Vd-i,k) ■ Vd,k{nd)- As Sn^ is 
non-zero there must be some kQ such that Vd,koiiT-d) 0- Define A : F"'' — >• F*^*^ to be the linear 
transformation defined by its action on the standard basis 



A{e. 



i4) 



_ '^d.fco(l) ^ 



1). 



i'd,kQ (n-d) 



°nd-l,d 



else 



Letting / denote the identity transformation, consider the tensor T' := (/ (8) • • • (8> / A){T) E 
F"i (g) • • •OF"'*. By LemmaEJl we observe that T' = [Si + ciSnJ ■ ■ ■ l-^n^-i + Cnd-i5'nJ5'„J, where 

By Lemma [A. 31 we have the simple tensor expansion T' = "^^.^i vi^k (8) • • • (8) Vd-i,k ® Avd^k- By 
construction, A{vd^ko) ~ ^d,koin'd) ' (^nd,d- Using Lemma [A. 41 we observe that the simple tensor 
'^''i.fco <8' • • • <8> Vd-i,kQ <^ Avd^ko non-zero entries only on the S„^-layer. 

We now define the linear transformation A' : F"** — t- F"'*^-'^ defined by 



A'ie,. 



iA 




Ud 



This will correspond to dropping the 5n^-layer. We can compose this with A to get A" = A' o A, 
defined by 



A"{ei 



'"d,kQ (nd) 



rid-l-d 



if i = 

else 



nd 



So now we take T" = (/(g)- • ■®I®A"){T). By Lemma|X6]we see that T" = [Si + ciS^^ ■ ■ ■ |5'„^-i + 
Cn^-iSn^- Further, we observe now that by construction A"{vd,ko) ~ ^- This leads to the simple 
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tensor expansion, 

r 

k=l 



Vd-ik ® A"vHk 



Vd-ik iXi A"vdh 



= vi^ko iX" • • • Vd-i,ko A"vd^ko + X] ^i-'^ 

k=l,k^ko 

r 

= vi^ko Vd-i,ko + ^ vi^k<^---<i 

k=l,k^ko 

r 

k=l,k^ko 

Therefore rank(r") < r — 1 = rank(T) — 1, and thus rank(r) > rank(T") — 1. 
The layer-reduction lemma will mostly be used via the following extension. 



□ 



Corollary B.2 (Iterative Layer- Reduction) . For layers Si,...,Sn^ G (8> • • • (g) F"'*-! with 
Si,...,Sm linearly independent (as vectors in the space ^^i---^d-i there exist constants Cij G F, 
i G {1, . . . , m}, j G {m + 1, . . . , 71,^}, such that 



(B.3) rank([S'i| . . . \SnJ) > rank 



Sm+l + ^ ^ Cj,m+l'S'i 



i=l 



+ m 



Proof. The proof is by induction on m. 

m = 1: This is Lemma [B.H up to reordering of the layers, with the observation that the singleton 
set {5*1} is linearly-independent iff Si is non-zero. The reordering of layers is justified by Lemma [A. 71 

m > 1: By the induction hypothesis we have that 



(B.4) rank([5i| . . . \SnJ) > rank 



m— 1 



i=l 



m—1 



i=l 



+ m — 1 



for the appropriate set of constants Cij. As the Si are linearly independent, Sm + YlT^i^ Ci^mSi is 



non-zero and so we can eliminate this layer from 
Lemma IB. II and consequently have 



m—1 



by 



(B.5) rank 



m—1 



Sm ~l~ ^ ^ Ci^rnSi 



1=1 

> rank 



m—1 



1=1 



m—1 



m—1 



Sm+1 + ^ ^ Ci^rn+lSi + Cm,m+1 I S-iYi + ^ ^ Cj,m'S'j 



1=1 



m—1 



■m,nd 



1=1 



' m-1 \ 

^ 1=1 / 



+ 1 



where the Cm,j are new constants. Now define 



(B.6) 



Cj,j ~l~ Cm,jCj,m if i ^ m 



else 



Combining Equations ()B.4p , (|B.5p , and ()B.6p yields the desired Equation (|B.3p . 
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□ 



Notice that by Lemma lA. 81 we can in fact use Lemma IB . 1 1 and Corollary IB .21 along any axis, not 
just the d-th one. 



Remark B.7. Lemma [B . 1 1 shows that the rank of T : ]^ 



F is at least 1 more than the rank 



of some T' : Ylj^i[n'A — )■ F, where n'- 



for all j 7^ jo, and n'- 



JO 



n 



'Jo 



1. In using this lemma. 



the quantity Y2j=i decreases by one. Therefore, we can never hope to apply this lemma more 



than Y2j=i many times, and thus using this lemma alone will never produce lower bounds larger 
than this quantity. Corollary IB. 21 simply applies Lemma IB. 11 so the same barriers apply. 

Appendix C. Proofs for Section [5] 



Proof of Lemma \5.S\ Notice that the left hand sides of Equation 15.31 and Equation 15.41 are equal. 
This follows from applying Corollary IA.9I twice (using that this corollary extends to layers along 
any axes, not just the d-th, by applying Lemma lA.Sp . once on the layers slicing the page vertically, 
and once on the layers slicing the page horizontally. Thus, it is enough to show Equation 15.31 

We now apply Corollarv lB.2l First, we use it on the layers slicing the page vertically and deriving 
that 



(C.l) 



rank 



In On 
On In 



o„ 



0„ 
0„ 



0„, 



0„ 
0„ 



> rank 



In 

c 



o„ 



0„ 
Ak 



+ n 



where C is an n x n matrix of field elements defined by the constants q j- of Corollary IB.2I Notice 
that the layers being dropped in the use of this corollary must be linearly independent. However, 
as they are the layers of [/n|0,i| • • • |0n] which slice the page vertically, they have exactly one 1 in 
the first ro and have entries elsewhere. As their non-zero entries are in different positions, 
they are linearly independent. Similarly, we can apply the corollary again on the remaining layers 
that slice the page horizontally to see that 



(C.2) 



rank 



In 


On 




On" 


c 


Al 




Ak 



^ > rank([C + C7'|^i| • • • \Ak]) + n 



where C is yet another n x n matrix of field elements produced by Corollary IB.2I We now invoke 
Corollarv IA.9I to observe that 

(C.3) rank([C + C'\Ai\ ■ ■ ■ \Ak]) > rank([^i| • • • \Ak]) 

Combining Equations ()C.ip . (|C.2p . and ()C.3P yields Equation (j5.3p and thus the claim. □ 

Proof of Theorem \5M ([I]): This is clear from construction. 

([2]): We first note that [lg2nj = [Ign -|- Ij = [IgnJ -|- 1. We first prove the upper bound, and 
then the lower bound. 

To see that rank(r„) < 2n — 2H{n) -|- 1 we observe that T„ has exactly this many non-zero 
entries. Denote this quantity r^. We proceed by induction on the recursive definition of the Sn,i- 
For n = 1, there is clearly exactly 2 • 1 — 2H{\) -|- 1 = 1 non-zero entry. For 2n > 1, r2n = r„ -|- 2n 
which by induction yields r2n = (2n — 2H{n) -|- 1) -|- 2n. Observing that H{n) = H{2n), we see 
that r2n = 2(2n) — 2H{2n) + 1. For 2n -|- 1 > 1, 'r2n+i = rn + 2n, which by induction yields 
f2n+i = (2re — 2H{n) -|- 1) -|- 2n. Noticing that H{2n -|- 1) = H{n) -|- 1 we have that r2n+i = 
2n - 2{H{2n + I) - I) + I + 2n = An + 2 - 2H{2n + I) + 1 = 2{2n + 1) - 2H{2n + + Thus, 
the induction hypothesis shows that rn = 2n — 2H{n) + 1 for all n, and thus upper-bounding the 
rank by this quantity. 

For the rank lower bound, we use Lemma 15.21 and induction on the recursive definition of the 
Sn,i- Clearly, rank(ri) > 1. Then for 2n > 1, rank(T2n) > rank(T„) -|- 2n, and for 2n -|- 1 > 



It is immaterial whether we call this a "row" or "column" , as no specific orientation of these tensors was chosen. 
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1, rank(r2„+i) > ra.nk{Tn) + 2n. These are exactly the same recurrences from the proceeding 
paragraph, and so they have the same solution: rank(T„) >2n — 2H{n) + 1. 
Combining these two bounds shows that rank(T„) = 2n — 2H{n) + 1. 

([3]): This is clear from the equations defining the Sn,i- D 

Proof of Corollary \5.1\ ([T]): This is clear from construction. 

([2|): Observe that in the construction of T^, the matrices S'^ j for i > [lg(n — 1)J + 1 are linearly 
independent. Thus, applying Corollarv lB.2l we see that 

rank(r^) > rank([5;,i| • • • |5;^Lig(n-i)J+i]) + ™ " (Llg(™ " 1)J + 1) 

where 

Sn—l,i—l Q 



for some arbitrary vectors q G F"^^. It follows from Corollarv IA.9I that we can drop the bottom 
row and last column of each of the Sn,i without increasing the rank, so that 

rank([5^_i| • • • |5^_Lig(„_i)j^ J) > rank([S'„_i,o| • • • \Sn-i,iig{n-i)\]) 

where the Sn-i,i-i are as defined in Theorem 15.61 and as such, rank([5„_i^o| ■ ■ ■ l'S'n-i,[ig(n-i)j]) = 
2{n — 1) + 2H{n — 1) + 1. Combining these inequalities yields the rank lower bound for T^. 

([3|): This is clear from the equations defining T^, and using the explicitness of the Sn^i^i^i as 
seen from Theorem 15.61 □ 




Appendix D. Algebraically-defined Tensors 

The results of this section will be field-specific, and so we no longer work over an arbitrary field. 

Lemma D.l. Let ¥q he the field of q elements. Consider nxn matrices Mi, . . . ,Mk over ¥q such 
that all non-zero linear combinations have full-rank. Then the tensor T = [Mi \ ■ ■ ■ |Mfc] has tensor 

rank at least —r — i-^n. 

Proof. The proof is via the probabilistic method, using randomness to perform an analogue of 
gate elimination. For non-zero c G F^, the summation c • M will nullify terms in a simple tensor 
expansion with some probability. This will in expectation reduce the rank. We then invoke the 
hypothesis that the result is full-rank, to conclude the bound on the original rank. 

Consider a minimal simple tensor decomposition T = "^21=1 Ui vi ® Wi. For c G F^, consider 

(notation-abused) dot-product {c,M), which can also be written as the matrix X]i=i'^«-^«- -^^y 
Lemma lA. 61 it can be seen that this is the image of T under the linear transformation I I A, 
where A is the linear transformation that sends the basis element Cj to Qei. Consequently, we have 
that (c, M) = (I (g) I (g) A)(T) = Yll=i Ui0Vi(d) (Awi). Noticing that Awi = (c, Wi), and that we can 
then treat this as a matrix instead of one-layer tensor, we see that (c, M) = Y2l=i{'^^''^i)^i ® ^i- 

Minimality implies that Ui ^ for all i. So for a fixed i, the set of c such that {c,Wi) = is a 
1-dimensional subspace by the Rank-Nullity theorem. Using that the field size is q, this shows that 



Pr [{c,m)^0] = 

cG„Ffe\{0} 




k k — 1 

Now define := {i\{c,Wi) ^ 0}. By linearity of expectation, Eg.g^pfe.^|(j|[|S'^|] = ~k'Li ^- Thus, 

k k — 1 —* 

there exits a non-zero cq such that jS^gl < qk^^i f- Therefore, we can write the matrix {c,M) as 
{c,M) = ^[=i(c, tt;j)uj ®Vi = l^jg5(c, ttJj)nj vi. The hypothesis on the Mj says that {c,M) is of 
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— * k k — 1 

full-rank, and therefore we have that n < rank((c, M)) < \S\ < ~k^_i f- As rank(T) = r, we have 



that rank(r) > Jj^ti n. □ 

Corollary D.2. Let he the field of q elements. Consider n x n matrices Mi, . . . , M„ over ¥g 
such that all non-zero linear combinations have full-rank. Then the tensor T = [Mi \ ■ ■ ■ \Mn] has 
tensor rank at least ^rr^ ~ RoSg^l ~ = '^qli ^ ~ ©(loggn). 

Proof. Let A; < n be a parameter, to be optimized over later. 

Notice that the hypothesis show that the matrices Mj are linearly independent and so Corol- 
lary [B2] shows that 

rank([Mi| • • • |M„]) > rank([Mi + M(| • • • \Mk + M^]) + {n - k) 

where the M- are linear combinations of the M^+i, . . . , M„. Thus, any non-zero linear combination 
of the {Mi + M-) is necessarily a non-zero linear combination of the Mj. In particular, this shows 
that any non-zero linear combination of the (Mi + M-) has full-rank. Thus, by Lemma ID. II 

k 1 

rank([Mi + M[\ • • • |Mfc + M^]) > J Sqk-i '' 
and so ^ 

rank([Mi| • • • |M„]) >n-k+ J SgLi "^ =' 

One can observe that k = log^ ijzjf^j maximizes /, but asymptotically it is sufficient to take 



k = \log„ n] . Then 



1 — q ^ 

f[k) = n-k + — n 

1 - Xjq 

n 1-1/n 

> n - log n — n 

\-\lq 

>n-riog,nl+^-^n-^-^n 

> n - I logg n I H -n 



1 0-1 



> -n - |loggn| 



^1 ' ' g-1 

As /(A;) lower-bounds the rank by the above, this establishes the claim. □ 

The above lemma and its corollary establish a property implying tensor rank lower bounds. We 
now turn to constructing tensors that have this property. Clearly we seek explicit tensors, and by 
this we mean that each entry of the tensor is efficiently computable. 

We first observe that the property can be easily constructed given explicit field extensions of the 
base field F. 

Proposition D.3. Let F 5e a field and f G F[x] he an irreducible 'polynomial of degree n. Then 
there exists n x n ¥ -matrices Mi, . . . , M^, such that all non-zero ¥ -linear combinations of the Mi 
have full-rank. Furthermore, the entries of each matrix are computable in algebraic circuits of size 
O(polylog(n)poly(||/||o)), where ||/||o is the number of non-zero coefficients of f . 

Proof. Let /(x) = Onx" + • • • + aix + oq. Recall that IK = F[x]/ (/) is a field, and because deg f = n, 
K is a n-dimensional F-vector space, where we choose 1 the basis. This gives an 

F-algebra isomorphism fi between K and a sub-ring M of the mxm F-matrices, where M is defined 
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as the image of /x. The map fj, is defined by associating a E K with the matrix inducing the Unear 
map /u(a) : F™' — )■ F™, where is the multiphcation map of a. That is, using that K = F"* we 
can see that the map /3 i— )• a/3 for /3 E IK is an F-hnear map, and thus defines /i(/3) over F"*. 

That the map is injective fohows from the fact that fJ.{a) must map 1 E IK to a E IfC, so 
a is recoverable from ^(a) (and surjectivity fohows be definition of M). To see the required 
homomorphism properties is also not difficult. As {a + 7)/3 = a/3 + for any a, ^,7 E K, this 
shows that /i(a+7) = ii{a)+fi{'y) as linear maps, and thus as matrices. Similarly, as {a'y)(3 = a(7/3) 
for any a, /3, 7 E IK it must be that fJ-ictj) = fi{a) fj,{'y) . That this map interacts linearly in F implies 
that it is an F-algebra homomorphism, as desired. 

In particular, this means that a E IK is invertible iff the matrix ^(a) EMC J?"^^" is invertible. 
As IK is a field, the only non-invertible matrix in M is /i(0). The F-algebra homomorphism means 
that for Oj E F and Oi E IK, the linear combination ^^ainlai) equals //(^ajaj) and so the matrix 
^aifi(ai) is invertible iff ^Ojaj 7^ 0. Thus, as l,x, . . . ,x"'~^ are F-linearly independent in IK, 
it follows that the matrix //(l),;^(x), . . . have that all non-zero F-linear combinations are 

invertible, as desired. 

We now study how to compute ^(x*). Observe that acting as a linear map on F", ^{x^) sends 
x^+j (mod /). To read off the x*^ component can be done with a lookup table to the coefficients of 
/, and thus in O(poly(||/||o)) size circuits. □ 

To make the above construction explicit, we need to show that the irreducible polynomial / can 
be found efficiently. We now cite the following result of Shoup |Sho90] . It says that we can find 
irreducible polynomials in finite fields in polynomial-time provided that the field size is fixed. 

Theorem D.4 ( jSho90| . Theorem 4.1). For any prime or prime power q, an irreducible polynomial 
of degree n in ¥q[x] can be found in time 0{po\y{nq)). 

Combining Corollarv ID.2| Proposition ID. 31 and Theorem ID. 41 we arrive at the following result. 

Corollary D.5. For any fixed prime or prime power q, over the field Fq there is a family of tensors 
Tn of size [n]^ such that 

(1) rank(r„) > - e(logg n) 

(2) On inputs n and {i,j,k) E [n]^, Tn{i,j,k) is computable in 0{po\y{nq)) 

Note that this is strictly worse than Corollary 15.71 in two respects. First, while this result 
asymptotically matches the lower bound of Corollarv 15.71 over F2, the above result is only valid over 
finite fields, and as the field size grows, the lower bound approaches 2n — o{n). This seems inherent 
in the approach. 

Further, the given construction is less explicit as computing even a single entry of the tensor 
might require examining all n of the coefficients in the irreducible polynomial /, preventing a 
0(polylog(n)) runtime. One method of circumventing this problem is to use sparse irreducible 
polynomials. In particular, we use the following well-known construction. 

Lemma D.6 ( [vL99j . Theorem 1.1.28). 0?;erF2[x], the polynomial 

f{x) = x^-^' +x^' +1 

is irreducible for any I > 0. 

Observe that this allows for much faster arithmetic in the extension field, and this leads to the 
following result when applying the above results. 

Corollary D.7. Over the field ¥2, there is a family of tensors Tn of size [n]^ defined for n = 2-3\ 
such that 

(1) rank(r„) > 3n - e(lgn) 
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(2) On inputs n and {i,j,k) G [n]^, Tn{i,j,k) is computable in 0(polylog(n)) 



Thus, this algebraic construction is also explicit, at least for some values of n. Also, this corollary 
is not limited to F2. Other constructions |GP97j are known over some other fields. However, unlike 
the results of Section [Sj it is not clear if better lower bounds exist for the tensors in this section. 
Indeed, we do not at present know non-trivial upper bounds for the tensors given here. 

Appendix E. Higher-Order Tensors 

In this section we investigate order-d tensors, particularly when d is odd. As Raz [RazlO ] shows, 
we can always "reshape" a lower-order tensor into a higher-order tensor without decreasing rank. 
Raz mentions this for reshaping an order-d tensor into an order-2 tensor (a matrix) and thus shows 
that there are explicit order-d tensors with rank n^'^^'^K We use our results for order-3 tensors to 
derive a better bound in the case when d is odd. 

We first state our reshaping lemma, keeping in mind that we again now work over an arbitrary 
field. 

Lemma E.l. Let T be an order-3 tensor of size [n"^] x [n'^] x [n]. Then define the order-{2d + 1) 
tensor T' of size [n]'^'^^^ by 

(d-1 d-1 
j=o j=o 

Then T' has rank at least rank(T). 

Further, ifT(-, •, •) is computable in time f{n), then T'{-, ...,•) is computable in time 0(poly(d)polylog(n)-|- 
fin)). 



Proof. First observe that the map (ii , . . . , z^) 1— t- 1 -|- X]j'=o (b'+i ~ 1)'^'' is a bijection from [n]** to [: 
as this is simply the base-n expansion. That this is map is computable in time 0(poly((i)polylog(n)) 
establishes the claim about efficiency. 

Now consider a rank r' decomposition of T' 

r' 

T' = ^Vl,l(E)---^Vl^2d+l 
1=1 

We now define ui^i G F"''. Via the bijection from above, we can write 



n 



d-1 



Ull 



1 + ^{ij+i - l)n^ j = vi,i{ii) ■ ■ ■ vi,diid) 
j=0 



and similarly we define m;_2 ^ by 

/ .1 \ 
ui,2 1 + "^(ij+id+i) - = vi,d+iiid+i) ■ ■ ■ Vl^2d{i2d) 

\ / 
, and we take ui ^ = vi^2d+i £ Thus we see that 

r' 

T = '^Ul,l (X" Ul,2 <^ Ul^s 

1=1 

by examining the equation pointwise, and thus r' > rank(T). The conclusion thus follows when 
taking r' = rank(T'). □ 
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The above lemma shows that rank lower bounds for low-order tensors extend (weakly) to rank 
lower bounds of higher-order tensors. We now apply this lemma to the tensor rank lower bounds 
of Section [5l It is possible to do similarly with the results of Section [Dl but a weaker conclusion 
would result as those lower bounds are weaker. 

Corollary E.2. For every d > 1, there is a family of {0,1} -tensors Tn of size [n]^'^^-'^ such that 
rank(T„) = 2n'^ + n — Q{d\gn). Further, given n and ii,. . . ,i2d+i, Tn{ii, ■ ■ ■ ,«2d+i) is computable 
in polynomial time (that is, in time 0(poly((i)polylog(n)) j. 

Proof. We first observe that the proof of Corollary 15.71 extends to give a family of tensors T„ with 
size [n] x [n] x [/(n)], where rank(T„) = 2n + f{n) — 0(lgn), where a;(lgn) < f{n) < n. Further, 
these tensors have their entries computed in 0(polylog(n)) time. 

Thus, this leads to tensors of size [n'^] x [nf^] x [n] of rank 2n'^ + n — G((ilgn). By Lemma lE. II 
we can then see that these tensors can be reshaped into the desired tensors, establishing the claim 
on the rank as well as the explicitness. □ 

Appendix F. Proofs for Section [6] 
We give proofs of various claims from Section [6l and examine the tightness of some of the results. 

Proof of Proposition \6. 1\ To apply Ben-Or's interpolation idea to tensors, we first note the connec- 
tion between tensors and polynomials. Consider the space of polynomials V := IFll-^^i^^litTo^, ■ ■ ■ , {^i''^^} {"={)] ^ 
that is, polynomials on the variables X^^^ that are set-multilinear with respect to the sets . 
One can call such a polynomial simple if it can be written as 

d 

i=i 

One can then define the rank(p), for p £ V, as the least number of simple polynomials needed to 
sum to p. One can observe that P is a [n]*^ tensor product space, and the notions of rank coincide. 
In this language, we seek to upper-bound the rank of the polynomial 

d(n-l) 
171=0 iiH hid=™ 

To implement Ben-Or's method, define the auxiliary polynomial P by 

i=l 

For fixed a, this polynomial is simple. When a is considered a variable, this polynomial has degree 
d{n — 1) in a. Further, the coefficient of a™ is 

iiA l-jd=m 

which corresponds exactly to tensors of the desired form. We can now interpret the auxiliary 
polynomials as polynomials in F({Xp^}jj)[a], the polynomial ring in the variable a over the 
field of rational functions in the X^''\ As |F| > d{n — 1), we can consider the evaluations 

{P{ai,{X{j)-}ij)}'l^l'^^ G ¥[{xl^^}ij] for distinct a; G F. Polynomial interpolation means that 
the coefficients Ca^{P) are recoverable from linear combinations of the evaluations of P. As the 
ai £ F, the (linear) evaluation map from the coefficients of P to the evaluations is defined by a 
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F-matrix. Therefore, the inverse of this map is also defined by an F-matrix. Specifically, there are 
coefficients Om,/ G F such that 

d(n-l) 
1=0 



Therefore, 




Cmam,l I P{ai,{Xi^^}i,j] 



m=0 



Thus, T is in the span of d{n — 1) + 1 simple polynomials. By moving the coefficients on the 
simple polynomials inside the product, this shows that T is expressible as the sum of d{n — 1) + 1 
simple polynomials. Using the above connection with tensors, this shows that the rank is at most 

We now prove that Proposition 16.71 is essentially tight. The proof uses Corollarv lB.2[ 

Proposition F.l. Let ¥ be a field. Let T be a tensor T : F such that 

1 ifh-\ \-id = n 

T{ii,. . . ,id) = {O ifk-\ \-id>n 

^ unconstrained else 

then rankF(r) > {d - l)(n - 1) + 1. 

Proof. The proof is by induction on d, using Corollary IB. 21 to achieve a lower bound. 
d = 1: As T 7^ 0, its rank must be at least 1, so the result follows. 

d > 1: Decompose T into layers along the d-th. axis, so that T = [Tq\ ■ ■ ■ |r„_i]. Observe that 
the hypothesis on T implies that for any linear combination S = Ylk=o ^i^i must be that 5 7^ 0. 
For if not, one may consider the smallest i such that Cj 7^ 0. Then S{n — i,0, - ■ ■ ,0) = Cj by the 
hypothesis on T and the construction of 5, which is a contradiction as q 7^ 0. 

Thus, Corollarv IB. 21 implies that rankF(T) > rankF(To + Yl^=i o-i^i) + (^^ — 1) for some ai G F. 
However, observing that T' = Tq + Yl^Zi ^^iTi is an order-(d — 1) tensor fitting the hypothesis of 
the induction we see that rankF(T') > (d — 2)(n — 1) + 1. Combining the above equations finishes 
the induction. □ 

The above proposition shows that Proposition 16.71 is nearly tight, because together they show 

that defining T : In^ F by r(n, . . . , i^) = [ii H \- = nj, we see that {d - l)(n -!) + !< 

rankF(r) < 1) + 1. 

Proposition 16 . 71 was done by interpolating a univariate polynomial. By interpolating multivariate 
polynomials one may obtain an upper bound for the rank over group tensors arising from the direct 
product of cyclic groups. However, the same result is derivable in a more modular fashion, which 
we now present. We start with the folklore fact that tensoring two tensors multiplies their rank 
bounds. 

Lemma F.2. Let ¥ he a field. Let T : [n]'^ — t- F and S : [mY ¥ be two tensors. Define 
(T^S): {[n] X [m])^ ^F by 

{T (g) S){{ii,i[), {id,i'd)) = T{ii,. . . ,id) ■ S{i[,.. . ,1^) 
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Then rankF(T ® S*) < rankF(r) Tank^{S). 



Proof. Suppose T = XlLi ®j=i^j,i and S = Ya'=i '^j'=i^'j',i'- Then T{ii, ...,id) = Y.i Wj 
and S{i[,..., i'^) = J2i' Uf Thus, {T0S){{ii,i[), {id, i'a)) equals Ujj' 



Hi,v Y[{j,j'){^j,i ® Thus, as for fixed 1,1' the tensor n(j,j')('^j,« simple (as 

a ([n] X [m])'^ — )■ F tensor), this shows the claim. □ 

We now apply this to the direct product construction of groups. 

Corollary F.3. Consider integers ni, . . . ,nm G Z>2 and consider the finite abelian group G = 
X • • • X Z„ . Let ¥ he a field with at least maxi{d{ni — 1) + 1) elements. Then, rank(T^) < 

Proof. First observe the relevant definitions imply that T^^ = T^^^^^. Thus, the claim 
follows directly from Corollarv 16.81 and Lemma |F.2[ □ 

We now recall the Structure Theorem of Finite Abelian Groups. 

Theorem F.4 (Structure Theorem of Finite Abelian Groups (see, e.g. |Art91j )). Let G be a finite 
abelian group. Then there are (not necessarily distinct) prime powers ni, . . . ,nm G Z>2 such that 

G = l^rix X • • • X Ijrim • 

This theorem shows that Corollary IF. 31 extends to general groups. One can get better bounds 
if more information is known about the group, of if results such as Theorem 15.51 apply, but the 
next result shows that even without such information group tensors from finite abelian groups have 
"low" rank. 

Proof of Corollarv \6. 91 Observe that using the Structure Theorem of Finite Abelian groups, we 
can apply Corollarv IF. 31 to G = Hi^i ^^i) and using that d(nj — 1) + 1 < (as d > 2) shows that 
rankF(Tj) < H "i- As m < Ig |G| and IG] = H ^i, the result follows. □ 

Proof of Lemma \6.1(K Define m := dim^lK. Thus, we can identify K = F™" as vector spaces, where 
we choose that 1 G K. is the first element in the F-basis for K.. This gives an F-algebra isomorphism 
^ between IK and a sub-ring M of the mxm F-matrices, where M is defined as the image of fi. The 
map fi is defined by associating x € K. with the matrix inducing the linear map n{x) : F™" — t- F*", 
where fj,{x) is the multiplication map of x. That is, using that IC = F™ we can see that the map 
y ^ xy for y G K is an F-linear map, and thus defines ^{x) over F™. 

That the map is injective follows from the fact that /i(x) must map 1 G IK to x G K, so 
X is recoverable from /i(x) (and surjectivity follows be definition of M). To see the required 
homomorphism properties is also not difficult. As {x-\-z)y = xy + zy for any x,y,z G IK, this shows 
that /i(x + z) = n{x) + ii{z) as linear maps, and thus as matrices. Similarly, as {xz)y = x{zy) for 
any x,y,z G IK it must be that fi{xz) = ^{x)^{z). That this map interacts linearly in F implies 
that it is an F-algebra homomorphism, as desired. 

Now consider a tensor T : [n^' — )• F with simple tensor decomposition T = ^^^"'^kCT") (^d^^g^.^^ 
over IK. First observe that if we define the map vr : IK — )• F defined by 



7r(x) 



X if X G F 
else 



then T = ^^^"^^kKCT) ^(^^d^^g.^^y -pj^^g ^^^^i /, TT{®j^^dj^i) is a tensor T/ : [n^ ^ F. 

We now show that rankf'(T;) < m'^~^. First observe that for x G F, /u(x) is a diagonal matrix. In 
particular, because we chose 1 G IK to the first element in the F-basis for IK, for x G F, 7r(x) is equal 
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to the (1, l)-th entry in fj.{x). Thus, it follows that Ti{ii, . . . , id) = (/i(ai^/(ii)) • • • fJ'{Sd,i{id)))i i- By 
expanding out the matrix multiplication we can see that Ti is expressible as 

m m 

Ti{ii,...,id) = X] X] /^(«i,'(^i))i,fci"/^(^2,/fe))fci,fc2 • • •/^(«rf-i,/(^d-i))fcd-2,fcd-i"/^(orf,K^rf))fcd- 

fci=l ka-i=l 

and just as in Theorem 16.51 we see that for fixed kj the summands are simple F-tensors, and thus 
rankF(T;) < m'^~^. 

Using the observation that T = ^^^"'^kC^) g^^id the above bound for the F-rank of Ti, we then 
see that rankF(r) < (dim^ IC)"'^-'^ rankK(T'), as desired. □ 

Appendix G. Proofs of Section [7] 
Proof of Theorem 7.3. m-rankp (T^^ ) > n'^~^: We remark that the following lower bound will only 

rely on the fact that T^^ is a permutation tensor, and no other properties. 

In monotone computation, there is no cancellation of terms. Thus, in a monotone simple tensor 
decomposition T = Yl^i=i '^^^ see that the partial sums T<rn = X^I^i successively cover 
more and more of the non-zero entries of T. We will show that in any monotone decomposition of 
Tg^, at most one non-zero entry can be covered by any T;, which implies that the monotone rank 
is at least the number of non-zero entries, which is n'^~^. 

We now prove that in any monotone simple tensor decomposition T^^ = Yl'i=i'^'j=i^j,i^ each 
simple tensor Ti := ^j^iSj^i can cover at most one non-zero entry of T^^. Suppose not, for 
contradiction. Then there is a simple tensor Ti that covers at least two non-zero entries (ii, . . . , id) 
and {i[, . . . , i'^) of T^^. However, these tuples must differ in at least one index, we we assume without 
loss of generality to be index 1, so that ii / i[. Consequently, we must have that ai_;(ii), aij{i'i) > 
(as all field constants are positive in monotone computation). As aj^i^ij) > for j > 1 (as 
Ti{ii,i2, . . . ,id) = 1), it must be that Ti{i[,i2, ■ ■ ■ ,id) = nj>i > 0- However, this 

is a contradiction. For now this positive number at Ti{i[,i2, ■ ■ ■ ,id) cannot be canceled out by 
other simple tensors in a monotone computation and we must have T^^{ii,i2, . . . ,id) = by the 
fact that this is a permutation tensor. Thus, it must be that each simple tensor in this monotone 
computation can only cover a single non-zero entry of T^^, which implies the lower bound by the 
above argument. □ 
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